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INTRODUCTION 


This  Final  Report  covers  work  done  at  the  Naval  Aerospace 
Medical  Research  Laboratory  during  fiscal  years  1990  and  1991. 

The  Joint  Working^ Group  on  Drug  Dependent  Degradation  in  Military 
Performance  ( JWGD  MILPERF)  was  established  for  the  purpose  of 
■'eveloping  and  testing  procedures  to  evaluate  the  effects  of 
chemical  defense  pharmaceutical  agents  on  military  performance. 
The  JWGD3  MILPERF  has  sponsored  the  performance  assessment- 
research  effort  at  the  Naval  Aerospace  Medical  Research 
Laboratory  (NAMRL)  that  has  evolved  from  independent 
contributions  by  various  scientific  disciplines  into  an  organized 
multidisciplinary  test  system  called  the  NAMRL  Multidisciplinary 
Performance  Test  Battery  (NMPTB)  .  Thu  NMPTB  has  been  designed 
for  assessing  aviator  performance  from  a  five-discipline 
perspective  (i.e.,  cognitive,  vestibular,  visual,  auditory,  and 
physiological) .  Other  products  of  sponsored  research  efforts 
from  NAMRL  are  computational  models  of  human  performance  in 
operational  tasks  and  in  laboratory  performance  tests. 

The  Vestibulo-Ocular-Ref lex  Performance  Evaluation  Test 
(VORPET)  is  one  of  the  major  component  tests  in  the  NMPTB.  It 
allows  assessment  of  the  type  of  head/eye  coordination  that  is 
required  in  a  variety  of  military  weapons  systems  (especially 
aircraft).  It  is  most  relevant  to  the  aviator  who  must  make 
large  shifts  in  gaze  during  his  scan  of  cockpit  instruments  and 
scenes  in  the  outside  environment  such  as  horizon,  targets,  and 
wingman  position  when  doing  formation  flying.  In  a  JWGD3  MILPERF 
sponsored  validation  study,  the  VORPET  results  indicated 
degradation  of  performance  as  a  result  of  antihistamine 
administrations,  indicating  that  the  VORPET  may  be  useful  as  a 
sensitive  NMPTB  performance  test. 

The  test  requires  the  subject  to  directly  face  one  display 
that  provides  visual  fixation,  and  upon  presentation  of  a 
simultaneous  visual  and  auditory  cue,  to  rapidly  rotate  his  head 
toward  the  second  display.  At  the  onset  of  the  cue,  four  digits 
are  presented  for  a  brief  exposure  time  on  the  second  display; 
the  subject  task  is  to  call  out  as  many  as  possible  of  the  four 
digits  that  he  can  correctly  identify  (presently,  the  subject 
response  is  manually  entered  into  the  computer's  keyboard  by  the 
operator) .  The  exposure  time  of  the  second  display  is 
appropriately  adjusted  (increased  or  decreased)  according  to  the 
accuracy  of  response  of  the  previous  trial.  The  procedure  is 
repeated  for  a  number  of  trials  with  the  role  of  the  first  and 
second  displays  sequentially  interchanged  resulting  in 
bidirectional  gaze  shifts.  The  end  result  is  a  single  threshold 
time  for  each  direction  of  gaze  shift,  and  a  grand  mean  threshold 
time  based  on  the  simple  mean  of  the  left-  and  right-directed 
gaze  shift  thresholds. 
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The  main  objectives  of  this  research  effort  for  the  years 
FY90  and  FY91  were: 

(a)  To  complete  and  refine  the  automation  of  test 
instructions  and  computerized  scoring  of  the  VORPET  test  using 
voice-actuated  subject  response  and  test  instructions. 

(b)  To  construct  two  to  four  (depending  on  cost,  space,  and 
acoustical  isolation  requirements)  automated  VORPET  test  stations 
and  install  them  in  NAMRL's  Mobile  Field  Laboratory  trailer  #5 
after  appropriate  modification  of  the  trailer  required  for 
additional  test  stations. 

(c)  To  conduct  baseline  validation  studies  such  as 
test/retest  reliability  studies. 

(d)  Perform  systematic  follow-on  studies  to  incorporate 
appropriate  additional  tests  derived  from  the  NAMRL 
Multi-disciplinary  Performance  Test  Battery,  which  has  been 
developed  for  assessing  performance  effects  of  chemical  defense 
antidote/pretreatment  drugs. 

In  order  to  accomplish  objective  (a),  the  VORPET(l)  test 
initially  designed  and  implemented  to  operate  in  the  BASIC 
language  running  on  a  Hewlett-Packard  based  digital  computer  was 
upgraded  to  operate  in  a  Zenith  Z-248  environment,  and  adapted  to 
use  a  voice-recognition  interface  in  order  to  automate  test 
administration.  The  automated  VORPET  was  then  compared  to  the 
manual  VORPET  method  of  data  entry  by  the  test  operator. 

Statistical  analysis  of  the  data  collected  from  both  method 
and  a  third  static  VORPET  test  indicate  that  the  automated  VORPE 
test  does  not  yield  the  same  threshold  measurements  and  that  it 
will  introduce  unknown  effects. 

State-of-the  art  tachnol  "  in  voice  recognition  needs 
further  improvement.  Higher  data  acquisition  sampling  rate  and 
more  efficient  algorithms  are  needed  in  order  to  provide  the 
accuracy  required.  The  use  of  IBM,  and  IBM  compatible  computers 
like  the  386  and  486  systems  with  a  much  higher  rate  of 
instruction  execution  (33  MHz)  in  combination  with  faster  Digital 
Signal  Processors  (DSP)  to  acquire  and  analyze  human  voice 
responses  (50  MHz) ,  and  more  efficient  algorithms  should  make 
acquisition  and  analysis  of  rapid  voice  responses  more  accurate. 

The  other  listed  milestones  were  not  completed.  Further 
work  needs  to  be  done  in  the  area  of  digital  signal  processing. 
There  is  a  need  for  the  use  of  voice  recognition  systems  in 
computer-based  testing  similar  to  the  one  needed  to  incorporate 
the  VORPET  test  as  part  of  the  NMPTB . 
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2 .  METHODS 


The  VORPET  initially  designed  and  implemented  to  operate  in 
the  BASIC  language  running  on  a  Hewlett-Packard  based  digital 
computer  was  upgraded  to  operate  in  a  Zenith  Z-248  environment. 
The  VORPET  software  programs  were  converted  from  the  BASIC 
language  to  the  C  language  and  transported  to  run  in  2-248  based 
IBM  AT  compatible  computer.  The  programs  were  modified  to 
incorporate  the  VOTAN  automatic  voice-recognition  system  in  order 
to  automate  test  administration. 

Implementation  of  the  software  program  for  administration  of 
the  VORPET  was  done  in  the  C  language  for  two  main  reasons:  (a) 
code  compactness ,  and  (b)  all  the  library  routines  to  interface 
to  the  VOTAN  voice-recognition  interface  are  available  only  for 
the  C  language. 

Next,  the  accuracy  of  the  new  automated  VORPET  test  was 
compared  to  that  of  the  manually  administered  version.  An 
experimental  test  design  was  prepared  to  measure  the  left-  and 
right-directed  gaze  shift  threshold  times  (in  milliseconds)  under 
three  different  test  conditions: 

Method  l.  VORPET  test  administration  with  the  subject 
directly  viewing  the  stimuli  digits  displayed  on  a  CRT,  with  no 
head/eye  motion  required,  and  using  only  the  voice  recognition 
system  to  collect  the  subject's  responses.  This  method  was 
considered  to  be  a  static  VORPET  test,  to  give  a  measure  of 
accuracy  of  the  voice  recognition  interface  system. 

Method  2 .  VORPET  test  administration  using  the  manual 
method  where  the  subject's  voice  response  collected  by  the  test 
operator  is  used  to  determine  the  thresholds. 

Method  3 .  VORPET  test  administration  using  the  VOTAN  voice 
recognition  system  to  collect  and  evaluate  the  subject's  voice 
responses  to  determine  the  thresholds. 

The  VORPET  under  the  three  different  test  conditions  was 
administered  using  two,  three,  and  four  stimuli  digits.  Both  the 
operator's  manually  entered  response  and  the  responses  collected 
by  the  voice-recognition  system  were  recorded  in  methods  2  and  3. 
Thirty-six  naval  aviator  students  waiting  for  flight  training 
from  the  Naval  Aviation  School's  Command  participated  in  the 
experiment. 

Figure  1  shows  the  flow-chart  diagram  of  how  the  VORPET  test 
was  administered.  The  test  was  configured  in  this  manner  to 
facilitate  execution  of  the  experimental  design  described  below. 

Test  Administration.  As  soon  as  hardware  and  software 
initialization  is  completed,  prerecorded  voice  instructions  are 
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given  to  the  subject  by  the  VOTAN  voice-recognition  system  (the 
voice  recognition  system  can  be  programmed  to  provide  voice 
instructions  or  messages)  about  the  test  to  be  administered. 

Next,  subject  and  test-related  information  are  entered  by  the 
test  operator. 

Prior  to  its  use  to  capture  and  to  record  the  subject's 
voice  responses,  the  VOTAN  voice-recognition  system  is  trained  to 
the  subject's  vocal  responses  (the  numbers  2  through  9  in  the 
case  of  the  VORPET  test)  that  are  used  throughout  testing. 
Instructions  are  given  to  the  subject  about  the  training  period. 

At  the  end  of  VOTAN' s  voice-recognition  training,  a  test  or 
evaluation  of  the  training  is  performed.  Ins\ ructions  are  given 
to  the  subject  about  the  evaluation.  At  the  end  of  the  training 
test,-  a  summary  of  percentage  accuracy  for  individual  numbers  2 
through  9  as  well  as  in  groups  of  four  numbers  presented  in 
random  order  is  displayed  on  the  CRT  screen  for  the  operator  to 
see.  Results  are  also  printed  and  stored  on  magnetic  media. 

The  operator  is  then  queried  by  the  program  about  the  need 
of  repeating  the  training/evaluation.  If  the  operator  responds 
affirmative,  training/evaluation  is  repeated,  otherwise  the 
VORPET  test  is  then  administered. 

There  were  three  methods  of  VORPET  test  administration: 

1  -  No  Head  Movement  Mode.  In  this  mode,  the  stimuli 
numbers  are  displayed  on  the  CRT  screen,  which  the  subject  views 
with  no  gaze  shifts  required.  In  this  mode,  the  voice- 
recognition  system  is  used  to  record  the  responses  and  to 
interpret  the  results,  which  are  used  to  determine  the  gaze 
thresholds  (VORPET  thresholds) . 

2  -  Manual  Data  Entry  Mode.  In  this  mode,  the  stimuli 
numbers  are  presented  in  the  standard  fashion  that  the  VORPET 
test  is  administered.  Both  voice-recognition  system  and  manual 
data  entry  by  the  operator  are  used  to  record  the  subject's 
responses.  However,  only  the  manual  data  entered  by  the  operator 
are  used  to  determine  the  VORTET  thresholds. 

3  -  Voice  Recognition  Mode.  In  this  mode,  the  stimuli 
numbers  are  presented  in  the  standard  fashion  that  the  VORPET 
test  is  administered.  Both  voice-recognition  system  and  manual 
data  entry  by  the  operator  are  used  to  record  the  subject's 
responses.  However,  only  the  responses  collected  by  the  voice 
recognition  system  are  used  to  determine  the  VORPET  thresholds. 

The  final  form  of  the  VORPET  will  be  the  same  as  that  shown 
by  the  flow-chart  in  Figure  1,  with  decision  making  for 
retraining  of  the  voice-recognition  system  to  be  computer 
automated . 
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Figure  1.  VORPET  Flow-Chart  Diagram 


Experimental  Design.  Thirty-six  subjects  were  administered 
the  VORPET  under  each  of  the  test  modes  described  above  and  for 
four,  three,  and  two  stimuli  digits.  The  sequence  in  which  the 
three  different  modes  were  administered  was  randomized  but 
balanced.  The  36  subjects  were  split  into  3  different  groups. 
Seven  subjects  had  missing  test  data. 

Hypothesis .  The  main  hypothesis  to  be  tested  is  that  the 
mean  of  gaze  thresholds  for  methods  two  and  three  are  the  same: 


Ha: 

Hi  =  h2 

Hi  *  H2 

The  mean  of  the  gaze  thresholds  are  the  same. 
The  mean  of  the  gaze  thresholds  are  different. 


and  that  the  introduction  of  the  automated  voice-recognition 
system  does  not  alter  the  measurements  of  both  right-  and  left- 
gaze  thresholds. 

Each  subject  was  given  a  total  of  32  trials  each  time  the 
VORPET  was  administered,  with  a  30-s  rest  interval  at  the  end  of 
the  16th  trial. 

Tables  I,  II,  and  III  illustrate  the  right-gaze  threshold 
means,  the  left-gaze  threshold  means,  and  the  grand-gaze 
threshold  means  (left  and  right  threshold  average) ,  respectively, 
for  each  of  the  36  subjects  tested  using  methods  1,  2,  and  3 
described  above,  when  4,  3,  and  2  digits  were  used  as  the 
stimuli . 


3.  RESULTS  AND  DISCUSSION 

The  VORPET  provides  a  measure  of  the  right-  and  left-gaze 
threshold  as  well  as  the  "grand  mean  threshold"  defined  to  be  the 
average  of  both  the  right-  and  left-threshold  every  time  a 
subject  is  tested. 

The  data  tabulated  in  Tables  I,  II,  and  III  were  analyzed 
with  the  aid  of  the  SuperANOVA ( 2 )  Statistical  Package  run  on  a 
Macintosh  digital  computer.  Analysis  results  include  Anova  Type 
III  Summary  Table,  Modeling,  Interaction  Plots,  and  a  Type  III 
Manova  table  of  the  following: 

(a)  the  variables  Four-Digits,  Three-Digits,  and  Two-Digits 
as  a  function  of  the  methods  used,  namely,  Method-1,  Method-2, 
and  Method-3 . 

(b)  the  variables  Method-1,  Method-2,  and  Method-3  as  a 
function  of  the  number  of  digits  used  as  stimuli,  namely,  Four- 
Digits,  Three-Digits,  and  Two-Digits. 
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Table  I  -  Mean  right-gaze  thresholds 

FINAL  THRESHOLD-RIGHT 
milliseconds 

Obs.  Four-Digits  Three-Digits  Two-Digits 

Method  Method  Method 

123123123 

001  573  516  546  545  460  999  . 

002  545  778  641  487  487  541 

003  487  641  545  . 

004  659  705  712  .  .  ' 

005  487  705  705  487  516  573  460  487  487 

006  605  516  641  487  487  514  460  460  575 

007  460  705  853  545  487  631  460  487  487 

008  460  541  487  460  460  516  460  460  460 

009  487  623  573  460  460  487  460  460  460 

010  545  596  750  514  460  641  575  487  516 

011  487  487  487  545  487  487 

012  460  623  742  460  460  487  460  487  460 

013  577  577  659  487  460  609  674  487  631 

014  514  516  609  487  460  516  487  460  460 

015  577  676  666  545  460  545  460  487  587 

016  487  605  609  .  487  596  487  460  545 

017  460  487  487  460  460  460  487  460  487 

018  460  573  575  514  596  750  487  460  637 

019  487  705  631  460  545  516  460  487  487 

020  545  744  825  516  541  577  487  460  596 

021  783  545  545  545  487  545  487  487  609 

022  516  607  668  460  487  487  487  487  516 

023  514  541  643  666  487  666  487  460  545 

024  545  744  712  487  545  514  487  487  573 

025  487  744  705  460  573  631  487  514  460 

026  750  596  712  516  573  641  575  460  516 

027  516  516  545  545  460  516  460  460  573 

028  641  516  674  631  487  605  460  460  460 

029  512  631  659  460  487  577  545  487  514 

030  487  516  514  460  460  487  487  460  487 

031  577  545  750  577  514  516  546  487  487 

032  488  573  705  516  577  666  487  596  545 

033  545  545  577  460  514  545  460  460  545 

034  596  862  712  487  595  541  545  487  487 

035  487  674  668  487  460  577  516  460  487 

036  .  596  605  R75  A  R7  c-7-7  Acn  c-TtT 


I 


Table  II  -  Mean  left-gaze  thresholds 
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Table  III  -  Mean  left  and  right  gaze  thresholds 


GRAND  MEAN  THRESHOLD 
milliseconds 


Obs . 

Four-Digits 

Method 

Three-Digits 

Method 

Two 

-Digits 

Method 

1 

2 

3 

1 

2 

3 

1 

2 

3 

001 

620 

515 

621 

561 

487 

999 

• 

• 

002 

„  16 

741 

654 

487 

555 

543 

- 

• 

• 

003 

501 

578 

593 

. 

. 

. 

* 

• 

t 

004 

573 

« 

628 

. 

. 

. 

* 

• 

• 

005 

487 

690 

650 

487 

515 

559 

460 

474 

474 

006 

655 

530 

593 

502 

474 

545 

460 

474 

560 

007 

488 

668 

857 

529 

474 

604 

460 

487 

516 

008 

460 

529 

502 

460 

460 

502 

460 

487 

460 

009 

487 

584 

560 

460 

460 

514 

460 

460 

460 

010 

545 

584 

679 

514 

474 

650 

574 

474 

516 

Oil 

487 

517 

502 

516 

487 

487 

• 

. 

. 

012 

474 

645 

743 

460 

474 

474 

460 

474 

460 

013 

610 

618 

650 

502 

488 

548 

671 

487 

648 

014 

529 

529 

657 

474 

460 

488 

501 

460 

534 

015 

561 

710 

689 

516 

488 

591 

460 

487 

502 

016 

516 

589 

735 

• 

502 

647 

474 

501 

545 

017 

474 

501 

562 

460 

488 

474 

474 

474 

474 

018 

502 

502 

689 

515 

584 

679 

474 

643 

608 

019 

460 

502 

487 

460 

530 

606 

474 

686 

668 

020 

502 

676 

825 

591 

514 

561 

501 

460 

541 

021 

694 

530 

53  0 

545 

474 

530 

474 

474 

620 

022 

502 

590 

668 

502 

514 

559 

487 

474 

502 

023 

561 

514 

594 

576 

514 

666 

474 

460 

545 

024 

516 

757 

728 

487 

576 

576 

474 

487 

530 

025 

502 

744 

727 

474 

584 

559 

516 

514 

474 

026 

708 

586 

708 

530 

557 

653 

673 

487 

530 

027 

516 

488 

516 

588 

460 

546 

517 

460 

545 

028 

609 

560 

641 

631 

514 

605 

460 

460 

460 

029 

502 

603 

742 

474 

516 

532 

502 

487 

530 

030 

502 

502 

514 

474 

460 

487 

474 

460 

474 

031 

561 

545 

647 

518 

501 

591 

546 

474 

514 

032 

488 

616 

674 

502 

546 

648 

487 

556 

556 

033 

502 

559 

660 

487 

501 

516 

487 

474 

516 

034 

586 

487 

502 

487 

584 

541 

529 

487 

502 

0'3 

501 

670 

621 

487 

474 

577 

488 

474 

576 

036 

# 

544 

67  5 

642 

487 

591 

546 

460 

560 

RESULTS 
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The  tabulated  results  and  the  interaction  plots  are  shown  in 
the  following  pages  and  in  the  indicated  order. 

Right  direction  gaze  thresholds.  The  following  results  are 
from  analysis  performed  on  the  data  in  Table  I. 

DIGITS 

Type  III  Gums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

202062.435 

101031.217 

13 . 516 

.  0001 

Residual 

88 

657777 . 675 

7474.746 

Dependent:  Four-Digits  (ms) 


NOTE:  17  rows  have  been  excluded  frcm  calculations  because  of 

missing  values 

Model  Summary 

Dependent:  Four-Digits  (ms) 

Count  91 
R  .485 
R-Squared  .235 
Adjusted  R-Squared  .218 

RMS  Residual  86.457 


df 

Sum  of  Squares 

Mean  Square 

F-Value 

P-Value 

Model 

2 

202062.435 

101031.217 

13 . 516 

.  0001 

Error 

88 

657777.675 

7474.746 

Total 

90 

859840.110 

Model  Coefficient  Table 
Dependent:  Four-Digits  (ms) 


Beta  Std.  Error  t-Test  P--Value 


651.806 

15.528 

41.976 

.  0001 

Methods 

method 

1 

-114 . 841 

22.335 

-5.142 

.  0001 

method 

2 

-40.742 

21.960 

-1.855 

.  0669 

method 

3 

0.000 

• 

• 

• 

L2 


Type  III  Sums  of  Squares 


Source 

df 

Sum  of  Squares 

Mean  Squar< 

2  F-Value 

P-Value 

Methods 

2 

73722.166 

36861.083 

11.865 

.0001 

Residual 

88 

273390.823 

3106.714 

Dependent:  Three-Digits  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Three-Digits  (ms) 

Count  91 
R  .461 
R-Squared  .212 
Adjusted  R-Squared  .19* 

RMS  Residual  55.738 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

73722.166 

36861.083 

11.865 

.  0001 

Error 

88 

2733390.823 

3106.714 

Total 

90 

347112.989 

Model  Coefficient  Table 
Dependent:  Three-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


562.839 

10.011 

56.223 

.0001 

Methods 

method 

1 

-58.046 

14 .399 

-4 . 031 

.  0001 

method 

2 

-61.774 

14 . 157 

-4 . 3636 

.0001 

method 

3 

0.000 

• 

• 

• 

1 


Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

29699.516 

14849.758 

7 . 520 

.0010 

Residual 

88 

173782.594 

1974.802 

Dependent:  Two-Digits  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Two-Digits  (ms) 

Count  91 
R  .  382 
R-Squared  .146 
Adjusted  R-Squared  .127 

RMS  Residual  44.439 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

29699.516 

14849.758 

7.520 

.  0010 

Error 

88 

173782. 5S4 

1974.302 

Total 

90 

203482.110 

Model  Coefficient  Table 
Dependent:  Two-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


Blf 

521.097 

7.981 

65.289 

— 

.  0001 

Methods 

method 

l 

-24.786 

11.480 

-2 . 159 

.  0336 

method 

2 

-43 . 645 

11.287 

-3.867 

.  0002 

method 

3 

0.000 

• 

• 

• 
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Cell  Means  of  Three-Digits  (ms)  Cell  Means  of  Four-Digits  (ms) 


Methods 


METHODS 


Type  III  Sums  of  squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Digits 

2 

24941.367 

12470.683 

2.634 

.0770 

Residual 

96 

454535.805 

4734.748 

Dependent:  Method-1  (ms) 


NOTE:  9  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  Method-1  (ms) 

Count  99 
R  .  228 
R-Squared  .052 
Adjusted  R-Squared  .052 

RMS  Residual  68.810 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

24941.367 

12470.683 

2.634 

.  0770 

Error 

96 

454535.805 

4734.748 

Total 

98 

479477.172 

Model  Coefficient  Table 
Dependent:  Method-1  (ms) 


Beta  Std.  Error  t-Test  P-Value 


537.429 

11.631 

46.207 

.  0001 

BHHI 

two  digits 

-38.016 

16.971 

-2.287 

.  0244 

KHI31B9 

-21.035 

16.696 

-1.260 

.  2108 

0.000 

• 

• 

• 

Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  £ .quare  F-Value  P-Valu e 


Digits 

2 

359972.304 

179986.152 

44.892 

.0001 

Residual 

96 

384896.242 

4009.336 

Dependent:  Method-2  (ms) 

NOTE:  9  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  Method-2  (ms) 

Count  99 
R  .  695 
R-Squarad  .483 
Adjusted  R-Squared  .473 

RMS  Residual  63.3119 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

359972.304 

179986.152 

44 . 892 

.  0001 

Error 

96 

384896.242 

4009.336 

Total 

98 

744868.545 

Model  Coefficient  Table 
Dependent:  Method-2  (ms) 


Beta  Std.  Error  t-Test  P-Value 


613 . 543 

10.703 

57.325 

.  0001 

Digits 

two  digits 

-136.091 

15.617 

-8.714 

.  0004 

three  digits 

-114 . 149 

15.364 

-7.430 

.  0001 

0.000 

• 

• 

• 
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Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Digits 

2 

251747.018 

125873.509 

17.206 

.  0001 

Residual 

96 

702291.608 

7315.538 

Dependent;  Method-3  (ms) 

NOTE:  9  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  summary 
Dependent:  Method-3  (ms) 

Count  99 
R  .514 
R-Squared  .264 
Adjusted  R-Squared  .249 

RMS  Residual  85.531 


df  Sum  of  Squares _ Mean  Square  F-Value  P-Value 


Model 

2 

251747.018 

125873 . 509 

17.206 

.  0001 

Error 

96 

702291.608 

7315.538 

Total 

980 

954.038.626 

Model  Coefficient  Table 
Dependent:  Method-3  (ms) 


Beta  Std.  Error  t-Test  P-Value 


ii 

643.771 

14.457 

44.529 

.0001 

isssm 

-122  675 

21.095 

-5.815 

.  0001 

three  digits 

-71.681 

20.753 

-3.454 

.0008 

four  digits 

0.000 

• 

• 

• 

Left  direction  gaze  thresholds.  The  following  results  are 
from  analysis  performed  on  the  data  in  Table  II. 


DIGITS 

Type  III  Sums  of  squares 


Source _ df  Sum  oi  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

260444.274 

130222 . 137 

15 . 658 

.0001 

Residual 

88 

731859.858 

8316.589 

Dependent:  Four-Digits  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Four-Digits  (ms) 

Count  91 
R  .  512 
R-Squared  .262 
Adjusted  R-Squared  .246 

RMS  Residual  91.195 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

260444.274 

130222 . 137 

15.658 

.  0001 

Error 

88 

731859.858 

8316.589 

Total 

90 

992304 . 132 

Model  Coefficient  Table 
Dependent:  Four-Digits  (ms) 


Beta  _  Std.  Error  t-Test  P-Value 


Intercept 

Methods 


660.065 _ 16  r  379 

-131.'/89  2J.560 


-66.903 

40.299 

-5.594 


.  0003 
.  0001 


method  1 
method  2 
method  3 


0.000 


23 . 164 


-2.888 


.  0049 


Type  III  Sums  of  Squares 

Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

87791.741 

43895.871 

12.541 

.0001 

Residual 

88 

308008 . 699 

3500.099 

Dependent:  Three-Digits  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  summary 

Dependent:  Three-Digits  (ms) 

Count  91 
R  .471 
R-Squared  .222 
Adjusted  R-Squared  .204 

RMS  Residual  59.162 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

87791.741 

43895.871 

12.541 

.0001 

Error 

88 

308008.699 

3500.099 

Total 

90 

395800.440 

Model  coefficient  Table 
Dependent:  Three-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


573.871 

10.626 

54 . 008 

.  0001 

Methods 

method 

1 

-65.492 

15.284 

-4 . 285 

.  0001 

method 

2 

-65.581 

15.027 

-4 .364 

.  0001 

method 

3 

0.000 

• 

• 

• 
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Type  III  Sums  of  Squares 

Source _ df  Sum  of  Squares  Mean  Square  T'-Value  P-Value 


rr -  -  - - 

Methods 

2 

26045.233 

13022 . 617 

2.994 

.  0552 

Residual 

88 

382760.525 

4349.551 

Dependent:  Two-Digits  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  TwcvDigits  (ms) 

Count  91 
R  .252 
R-Squared  .064 
Adjusted  R-Squared  .042 

RMS  Residual  65.591 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

26045.233 

13022 . 617 

2.994 

.0552 

Error 

88 

382760.525 

4349.551 

Total 

90 

408805.758 

Model  Coefficient  Table 
Dependent:  Two-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


532 . 581 

11.845 

44.962 

.  0001 

Methods 

method 

1 

-29.753 

17 . 038 

-1.746 

0843 

method 

2 

-39.387 

16.752 

-2.351 

.  0209 

method 

3 

0.000 

• 

% 

24 


Methods 
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METHODS 


Typo  III  Sums  of  Squares 

Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Digits 

2 

14592.055 

7296.028 

1.950 

.  1479 

Residual 

95 

355469.424 

3741.783 

Dependent:  Method-1  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  Method-1  (ms) 

Count  98 
R  .  199 
R-Squared  .039 
Adjusted  R-Squared  .019 

RMS  Residual  61.170 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

14592.055 

7296.028 

1.950 

.  1479 

Error 

95 

355469.424 

3741.783 

Total 

97 

370061.480 

Model  Coefficient  Table 
Dependent:  Method-1  (ms) 


Beta  Std.  Error  t-Test  P-Value 


530.029 

10.491 

50.524 

.0001 

Methods 

-28.158 

15.191 

-1.854 

.0669 

-22.332 

14.948 

-1.494 

.1385 

0.000 

• 

• 

• 
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Typo  III  Sums  of  Squares 

Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Digits 

2 

186509.326 

93254.663 

21.503 

.  0001 

Residual 

95 

412002.276 

4336.866 

Dependent:  Method-2  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  Method-2  (ms) 

Count  98 
R  .  558 
R-Squared  .312 
Adjusted  R-Squared  .297 

RMS  Residual  65.855 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

186509.326 

93254 . 663 

21.503 

.  0001 

Error 

95 

412002.276 

4336.866 

Total 

97 

598511.602 

Model  Coefficient  Table 
Dependent:  Method-2  (ms) 


Beta  Std.  Error  t-Test  P-Value 


592.794 

11.294 

52.487 

.  0001 

Methods 

-99 . 601 

16.354 

-6.090 

.  0001 

-81.734 

16.093 

-5.079 

.  0001 

0.000 

• 

• 

• 
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Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Val u e 


2 

245475.882 

122737 . 941 

12.823 

.0001 

Residual 

95 

909286.526 

9571.437 

Dependent:  Method-3  (ms) 

NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  summary 
Dependent:  Method-3  (ms) 

Count  98 
R  .461 
R-Squared  .213 
Adjusted  R-Squared  .196 

RMS  Residual  97.834 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

245475.382 

122737.941 

12.823 

.  0001 

Error 

95 

909286.526 

9571.437 

Total 

97 

1154762.408 

Model  Coefficient  Table 
Dependent:  Method-3  (ms) 


Beta  Std.  Error  t-Test  P-Value 


654 . 088 

16.778 

38.984 

.0001 

Methods 

-121.508 

24.295 

-5 . 001 

.0001 

-74 . 603 

23.907 

-3 . 121 

.0024 

0.000 

• 

• 

• 

‘  «  Interaction  Plot 

i  Effect:  Digits 

\  Dependent:  Method-1  (ms) 

l  With  Standard  Deviation  error  bars. 


Average  left  and  right  direction  gaze  thresholds.  The 
following  results  are  from  analysis  performed  on  the  data  in 
Table  ill. 

DIGITS 

Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

182148.316 

91074 . 158 

13 .956 

.  0001 

Residual 

86 

561237.729 

6526.020 

Dependent:  Four-Digits  (ms) 


NOTE:  19  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Four-Digitr  (ms) 

Count  89 
R  .495 
R-Squared  .245 
Adjusted  R-Squared  .227 

RMS  Residual  80.784 


df _ Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

182148.316 

91074 . 158 

13 . 956 

.  0001 

Error 

86 

561237.729 

6526.020 

Total 

88 

743386.045 

Model  Coefficient  Table 
Dependent:  Four-Digits  (ms) 

Beta  Std.  Error  t-Test  P-Value 


Type  III  Sms  of  squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

75572.362 

37786.181 

16.269 

.  0001 

Residual 

86 

199742.694 

2322.589 

Dependent:  Three-Digits  (ms) 

NOTE:  19  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Three-Digits  (ms) 

Count  89 
R  .524 
R-Squared  .274 
Adjusted  R-Squared  .258 

RMS  Residual  48.193 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

75572.362 

37786.181 

16.269 

.  0001 

Error 

86 

199742.294 

2322.589 

Total 

88 

275315.056 

Model  Coefficient  Table 
Dependent:  Three-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


576.6 

8.799 

64.508 

.0001 

Methods 

method 

1 

-60.979 

12.550 

-4.859 

.0001 

method 

2 

-62 .267 

12.443 

-5.004 

.0001 

method 

3 

0.000 

• 

• 

• 

Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Methods 

2 

21833.685 

10916.842 

3.772 

.  0552 

Residual 

86 

2489900.225 

2894 . 189 

Dependert:  Two-Digits  (ms) 


NOTE:  17  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  summary 

Dependent:  Two-Digits  (ms) 

Count  89 
R  .284 
R-Squared  .081 
Adjusted  R-Squared  .059 

RMS  Residual  53.798 


df  Sum  of  Squares _ Mean  Square  F-Value  P-Value 


Model 

2 

21833.685 

10916.842 

3.772 

.  0269 

Error 

86 

248900.225 

2894.189 

Total 

88 

270733.910 

Model  Coefficient  Table 
Dependent:  Two-Digits  (ms) 


Beta  Std.  Error  t-Test  P-Value 


528 . 000 

9 . 822 

53.757 

.0001 

Methods 

method 

1 

-28.793 

14 . 101 

-2.055 

.0429 

method 

2 

-36.133 

13 . 890 

-2.601 

.  0109 

method 

3 

0.000 

• 

• 

• 

Methods 


Interaction  Plot 
Effect:  Methods 
Dependent:  Two-Digits  (ms) 

With  Standard  Deviation  error  bate. 


Method-1  Method-2  Method-3 


Methods 


Type  III  MANOVA  Table 
Effect:  Methods 

S  2 

M  0.000 

N  41.000 

Value  F-Value  Nuiu  DF  Den  DF _ P-Value 


METHODS 


Type  III  Sums  of  Squares 


Source 

df 

Sum  of  Squares 

Mean  Square 

F-Value  P-Value 

Digits 

2 

20832.391 

10416.195 

3.481 

.  0349 

Residual 

93 

278298 . 849 

2992.461 

Dependent:  Method-1  (ms) 

NOTE:  12  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  summary 
Dependent:  Method-1  (ms) 

Count  96 
R  .264 
R-Squared  .070 
Adjusted  R-Squared  .050 

RMS  Residual  54.703 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

20832.391 

10416.195 

3.481 

.0349 

Error 

93 

278298.849 

2992.461 

Total 

95 

299131.240 

Model  Coefficient  Table 
Dependent:  Metho^.-l  (ms) 


Beta  Std.  Error  t-Test  P-Value 


533 . 059 

9.302 

56.820 

.  0001 

Methods 

-34 . 692 

13.703 

-2.532 

.  0130 

-25.059 

13.473 

-1.860 

.  0661 

mrnrnm 

0 . 000 

• 

• 

• 

Type  III  Sums  of  Squares 


Source _ df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Digits 

2 

172872.302 

86436.151 

23.950 

.0001 

Residual 

93 

335636.65 6 

3608.996 

Dependent:  Method-2  (ms) 

NOTE:  12  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 
Dependent:  Method-2  (ms) 

Count  96 
R  .583 
R-Squared  .340 
Adjusted  R-Squared  .326 

RMS  Residual  60.075 


df  Sum  of  Squares  Mean  Square  F-Value  p-Value 


Model 

2 

172872.302 

86436.151 

23.950 

.  0001 

Error 

93 

335636.656 

3608.996 

Total 

95 

508508.958 

2 

Model  Coefficient  Table 
Dependent:  Method-2  (ms) 


Beta  Std.  Error  t-Test  P-Value 


587.029 

10.303 

56.978 

=  ==i  ^'= 

.0001 

Methods 

-95.163 

15.048 

■  6.324 

.0001 

three  djgits 

-81.186 

14 .796 

-5.487 

.0001 

four  digits 

0.000 

• 

• 

• 

Type  III  Sums  of  Squares 


Source 

df 

Sum  of  Squares 

Mean  Square 

F-Value 

P-Value 

Digits 

2 

-  ■ 

198594 . 142 

99297.071 

14 . 082 

.0001 

Residual 

93 

655753.816 

7051.116 

Dependent:  Method-3  (ms) 

NOTE:  12  rows  have  been  excluded  from  calculations  because  of 

missing  values 


Model  Summary 

Dependent:  Method-3  (ms) 

Count  96 
R  .482 
R-Squared  .232 
Adjusted  R-Squared  .216 

RMS  Residual  83.971 


df  Sum  of  Squares  Mean  Square  F-Value  P-Value 


Model 

2 

198594 . 142 

99297.071 

14 . 082 

.  0001 

Error 

93 

655753.816 

7051.116 

Total 

95 

854347.958 

Model  Coefficient  Table 
Dependent:  Method-3  (ms) 


Beta  Std.  Error  t-Test  P-Value 


638.824 

14.401 

44.360 

.  0001 

Methods 

-110.824 

21.034 

-5.269 

.0001 

-63 .511 

20.682 

-3 . 071 

.  0028 

0.000 

• 

• 

• 

Cell  Means  of  Method-2  (ms)  Cell  Means  of  Method-1  (ms) 


Interaction  Plot 
Effect:  Digits 

Dependent:  Method-1  (ms) 

With  Standard  Deviation  error  bars. 


Two-digits  Three-Digits  Four-Digits 


Digits 

Interaction  Plot 
Effect:  Digits 

Dependent:  Method-2  (ms) 

With  Standard  Deviation  error  bars. 


Two-digits  Three-Digits  Four-Digits 


Digits 
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Wilks'  Lambda 


Roy's 

Greatest  Root 

Hotelling- 
Lawley  Trace 


Pillai  Trace 


DISCUSSION 


Analysis  of  variance  performed  on  the  response  data 
indicates  that  threshold  measurements  obtained  using  Methods-2 
and  -3  are  significantly  different.  Thus  the  null  hypothesis 
that  administration  of  the  VORPET  using  the  automated  voice- 
recognition  system  will  yield  the  same  threshold  measures  as 
those  of  the  manual  test  administration  should  be  rejected. 

Interaction  plots  of  the  variable  Method-1  for  the  three 
analyses  show  varied  effect  results  as  a  function  of  stimuli 
digits  used.  For  the  right-directed  gaze,  analyses  indicate  no 
significant  difference  as  a  function  of  the  number  of  digits 
used.  Fox-  the  left-directed  and  the  average  gazes,  analyses 
indicate  a  significant  effect  as  a  function  of  digits  at  the  p  = 
0.07  and  p  =  0.03  level.  At  a  p  <  .01,  results  indicate  that 
there  is  no  significant  difference  in  the  means,  but  it  could  be 
because  the  standard  deviation  is  relatively  large.  The  use  of 
the  voice-recognition  system  should  have  yielded  the  same 
threshold  measure  of  460  ms  regardless  of  the  number  of  stimuli 
digits  used,  when  administering  the  VORPET  by  Method-1,  since  the 
subjects  looked  directly  at  the  stimuli  digits  presented  on  the 
CRT  with  no  head  movement.  The  interaction  plots  should  have 
shown  a  horizontal  line  (the  threshold  value)  as  a  function  of 
the  number  of  digits  used  during  administration  of  the  VORPET. 

Inspection  of  the  interaction  plots  and  the  AhuVA  results 
for  the  case  when  Digits  is  the  dependent  variable  and  Methods  is 
the  effect  or  independent  variable,  the  mean  threshold  for  Four- 
Digits  (right,  left,  or  grand  mean)  is  lowest  with  Method-1,  next 
higher  with  Method-2,  and  highest  with  Method-3,  with  a 
significant  difference  between  the  means  of  Method-2  and  -3. 

With  Three-digits,  the  mean  threshold  is  about  the  same  for 
Method-l  and  -2  and  substantially  higher  for  Method-3.  Using 
Two-Digits,  the  mean  threshold  for  Method-2  is  lower  than  that 
for  Method-1  and  substantially  lower  as  compared  to  that  of 
Method-3.  In  ail  cases,  the  means  obtained  from  Method-2  and  -3 
are  significantly  different,  indicating  that  the  null  hypothesis 
should  be  rejected. 

Comparison  of  the  interaction  plots  and  the  ANOVA  results 
for  the  variables  Method-2  and  Method-3  indicates  an  increasing 
trend  in  the  threshold  response  as  a  function  of  the  number  of 
digits  used  as  the  stimulus.  The  increased  trend  is  more  linear 
for  variable  Method-3  than  for  Method-2. 
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4.  CONCLUSIONS 

The  threshold  measures  for  right-  and  left-directed  gaze 
shifts  obtained  by  Methods-2  and  -3  were  statistically  different. 
This  means  that  the  voice-recognition  system  as  an  alternative 
use  to  the  manual  method  to  automate  the  administration  of  the 
NMPTB  tests,  is  not  satisfactory. 

In  addition,  the  measures  of  gaze  obtained  with  method  1 
differ  significantly  with  the  number  of  digits  (four,  three,  and 
two)  used. 

Full  automation  of  the  VORPET  cannot  be  accomplished  at  this 

time . 


The  state-of-the-art  technology  in  voice  recognition  needs 
further  hardware  and  software  improvement.  Higher  data 
acquisition  sampling  rate  and  more  efficient  algorithms  are 
needed  in  order  to  provide  the  accuracy  required.  The  use  of  IBM 
and  IBM-compatible  computers,  like  the  386  and  486  systems  with  a 
much  higher  rate  of  instruction  execution  (33  MHz),  in 
combination  with  faster  Digital  Signal  Processors  (DSP)  to 
acquire,  analyze  human  voice  responses  (50  MHz),  and  more 
efficient  algorithms,  should  make  acquisition  and  analysis  of 
rapid  voice  responses  more  accurate. 

The  other  listed  milestones  were  not  completed.  Further 
work  needs  to  be  done  in  the  area  of  digital  signal  processing. 
There  is  a  need  for  the  use  of  voice-recognition  systems  in 
computer  based  testing  similar  to  the  one  needed  to  incorporate 
the  VORPET  as  part  of  the  NMPTB. 
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